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Segmentation  using  Multispectral  Adaptive  Contours 

Introduction 

Vector- valued  images  provide  data  in  a  form  of  vector-pixels.  Multi  or  hyperspectral  images  are 
particular  cases  of  vector-valued  images,  which  consist  of  a  stack  of  two-dimensional  images,  each 
measured  at  different  wavelength  band.  A  simple  case  of  multispectral  images  is  a  color  image, 
consisting  of  red,  green,  and  blue  (RGB)  channels.  Hyperspectral  images  are  another  example,  which 
often  contain  more  than  hundred  bands.  Multispectral  images  have  been  studied  mainly  in  target 
detection  and  remote  sensing  scenarios. 

Active  contours  have  been  used  for  image  segmentation  since  the  first  introduction  of  snakes  by  Kass  et 
al.  [11].  The  basic  idea  is  to  evolve  a  contour  C,  subject  to  constraints  from  a  given  image  /,  in  order  to 
detect  objects  in  the  image.  For  instance,  starting  with  a  contour  around  the  object  to  be  detected,  the 
contour  moves  towards  its  normal  and  has  to  stop  on  the  boundary  of  the  object.  The  evolution  of  the 
contour  is  obtained  by  minimizing  an  energy  function  with  respect  to  C,  and  the  minimizer  is  given  by 
the  boundary  of  the  object.  The  classical  approaches  use  the  image  gradient  to  locate  the  boundary  of 
the  object.  In  the  problems  of  contour  evolution,  the  level  set  method  and  the  motion  by  mean  curvature 
of  Osher  and  Sethian  [18]  have  been  used  extensively,  because  they  allow  for  automatic  topological 
changes.  Moreover,  the  discretization  of  the  problem  is  made  on  a  fixed  rectangular  grid.  Active 
contours  provide  segments  in  a  form  of  closed  curves  (continuous  boundaries),  while  local  filtering 
based  edge  detectors,  i.e.  Canny  [1]  or  Sobel  operators,  often  provide  discontinuous  boundaries. 
However,  active  contours  have  a  few  disadvantages;  (a)  Active  contours  may  converge  to  a  local 
minimum  instead  of  the  global  minimum  if  the  initial  contour  is  placed  at  an  improper  position.  This 
problem  makes  it  difficult  to  apply  active  contours  in  a  complex  image,  particularly  a  textured  image. 
This  problem  also  increases  the  impact  of  noise,  (b)  The  computational  cost  is  higher  than  other 
segmentation  methods.  Since  multispectral  image  processing  has  the  same  drawback,  the  application  of 
active  contours  on  vector- valued  images  is  a  difficult  task. 

The  intent  of  this  research  is  to  investigate  the  performance  of  active  contours  as  an  image  segmentation 
tool  for  vector-valued  images,  particularly  multispectral  images.  A  review  of  published  methods  has 
been  performed,  along  with  numerical  experiments.  In  this  report,  we  divided  active  contours  into  two 
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groups;  (a)  geometry-  or  boundary-based  models  and  (b)  statistics-  or  region-based  models.  Boundary- 
based  active  contour  models  use  the  image  gradient  to  stop  the  evolution  of  contours  on  the  desired 
boundary.  Two  boundary-based  active  contour  models  are  reviewed  in  this  report;  geometric  active 
contours  proposed  by  Caselles  et  al.  [2]  and  by  Malladi  and  Sethian  [13,  14]  and  geodesic  active 
contours  proposed  by  Caselles  et  al.  [3,  4,  22,  23].  Region-based  active  contour  models  use  stochastic 
information  of  segments  to  stop  the  contour  evolution  on  the  desired  boundary.  Two  region-based  active 
contour  models  are  reviewed  in  this  report;  one  proposed  by  Chan  and  Vese  [5,  6,  7,  8]  and  the  other 
proposed  by  Rousson  and  Deriche  [21]. 

Most  active  contour  models  applicable  to  vector- valued  images  [7,  22,  23,  25]  evolve  the  contour  based 
on  the  sum  of  energy  terms  calculated  in  each  frame.  Since  hyperspectral  images  may  have  up  to 
hundreds  of  bands,  this  accumulation  process  may  average  out  important  features  if  they  are  observable 
only  in  a  limited  number  of  bands.  For  instance,  if  a  set  of  multispectral  images  of  a  car  are  taken  at 
visible  bands  and  infrared  bands,  the  hot  spot  of  an  engine  is  observed  only  in  infrared  (IR)  bands. 
Taking  the  sum  of  energy  terms  along  the  dimension  of  wavelength  may  remove  the  hot  spot  by 
smoothing  effect.  We  did  an  experiment  of  extracting  a  stack  of  prime  contours,  which  are  evolved  from 
a  group  of  frames  instead  of  the  whole  frames.  This  new  strategy  divides  a  given  hyperspectral  image 
into  a  group  of  multispectral  images  with  lower  number  of  bands,  based  on  the  correlation  between 
frames.  A  set  of  active  contours  -  the  prime  contours  -  are  evolved  from  each  of  the  multispectral 
images.  Final  segmentation  is  achieved  from  these  prime  contours.  This  approach  makes  it  possible  to 
avoid  the  undesirable  smoothing  effect  in  hyperspectral  image  segmentation. 
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Contour  evolution 

Most  active  contours  move  towards  their  interior  normal  and  stop  on  the  boundary  of  the  object.  In  most 
of  active  contours,  the  force  to  stop  the  contour  is  obtained  by  minimizing  an  energy  function,  and  the 
force  to  move  the  contour  is  based  on  the  mean  curvature.  In  this  section,  we  briefly  discuss  these  two 
topics. 

Energy  minimization;  snakes 

Let  a  contour  be  normalized  to  have  a  length  of  one  as 

C(5):[0,1]^E".  (1) 

The  classical  snake  method  associates  the  evolution  of  a  contour  C  with  an  energy  function'  given  by 

1  1  1 

E{C)  =  aJ|C'(5)f  ds  +  -/l||v/(C(5))|t/5 

0  0  0  (^) 

=  £,.(c)+£,.,(c) 

where  a,  p,  X  are  real  positive  constants  [11].  The  first  two  terms  (internal  energy)  control  the 
smoothness  of  the  curve.  The  first-order  term  with  a  makes  the  snake  acts  like  a  membrane  (i.e.  resists 
stretching),  while  the  second-order  term  with  p  makes  the  snake  acts  like  a  thin  plate  (i.e.  resists 
bending).  The  third  term  (external  energy)  with  X  attracts  the  contour  towards  the  boundary  of  the  object. 
By  minimizing  this  energy  function,  we  are  trying  to  locate  the  contour  at  the  points,  which  maximize 
|V/|,  acting  as  an  edge-detector,  while  keeping  a  smoothness  in  the  contour. 

Classical  snakes  have  a  few  disadvantages;  (a)  they  cannot  detect  more  than  one  object  because  the 
contour  cannot  merge  or  break,  (b)  Since  snakes  make  use  of  only  local  information,  they  may  not  find 
the  boundary  of  the  object  if  the  initial  position  of  the  contour  is  too  far  from  the  object. 

Curvature  motion;  level  sets 

Let  a  metric  function  be  defined  as 

(3) 


'  This  energy  function  is  simplified  particularly  for  the  image  segmentation.  We  are  interested  in  closed 
curves  (contours)  only. 
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The  contour  C  can  be  represented  implicitly  via  this  metric  function  by 


C(j')  =  {(x,;;)  :(l)[x,y,t)  =  O} 


(4) 


and  the  evolution  of  the  contour  is  given  by  the  zero-level  contour  at  time  t  of  the  function  A 

common  way  to  initiate  a  level  set  function  is  a  signed  distance  to  the  nearest  pixel  on  the  contour.  The 
deformation  of  the  contour  is  given  by 


(5) 


^^{x,y)  =  ±D{yi,y),C) 

where  F  is  the  speed  function  of  deformation  and  D{a,  b)  is  the  Euclidean  distance  between  two  pixels 
[18].  A  common  example  of  the  speed  function  is 

F{^k^  =  v  +  sk  (6) 

where  v  is  a  constant  speed  term  to  accelerate  the  motion  of  a  contour,  and  s  controls  the  relative 
importance  of  the  curvature  motion.  The  unit  normal  vector^  and  the  mean  curvature^  can  be  directly 
calculated  from  the  level  set  function. 


n  = 


(7) 


K  =  div 


_  (l>Jy  -  +  KF 

where  ^  and  (j)  xx  denote  the  first-  and  second-order  partial  derivatives  of  (j)  respect  to  x. 


(8) 


The  level  set  method  has  a  few  advantages  over  the  classical  snakes;  (a)  it  allows  for  automatic 
topological  changes,  i.e.  merging  and  breaking,  (b)  The  discretization  of  the  problem  is  made  on  a  fixed 
rectangular  grid.  However,  the  computational  cost  is  high  because  the  dimension  of  the  metric  function 
to  be  updated  is  equal  to  the  dimension  of  the  image. 


^  Some  literature  [14,  16]  defined  the  unit  normal  as  n  =  -V^/|V^|  instead  of  V^/|V^|  depending  on  the 
direction  the  normal  vector  towards. 

^  In  the  case  above,  curvature  is  defined  as  k=  -V-(V^/|V^|). 
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Boundary  (geometry)  based  active  contours 


Boundary-based  active  contour  models  use  the  image  gradient  to  stop  the  contour  evolution  on  the 
desired  boundary.  The  stopping  function  can  be  defined  as  a  positive  and  decreasing  function,  such  as 

s(i{x,y))  =  Y-^r — ^  (9) 


or 


(10) 


depending  on  the  image  gradient 

f(l{x,y))  =  \VG„*l{x,y)^  (11) 


where  is  a  smoother  version  of  1.  The  function  g(7)  is  positive  in  homogeneous  regions,  and  zero  at 
the  edges 


j/^O  g^l 

g^O' 


(12) 


Geometric  active  contour 

Caselles  et  al.  [2]  and  Malladi  et  al.  [13,  14]  proposed  geometric  active  contour  models  based  on  the 
mean  curvature  motion  given  by 

^  =  g{l){^  +  v)\V^\.  (13) 

where  v  is  a  positive  constant  pushing  the  contour  towards  the  object.  The  zero  level  contour  moves  in 
the  normal  direction  with  speed  +  v),  and  therefore  stops  on  the  desired  boundary,  where  g(I) 

vanishes. 

Geometric  active  contours  have  a  few  disadvantages;  (a)  since  the  stopping  function  use  the  image 
gradient,  boundary-based  active  contours  make  use  of  only  local  information  like  the  classical  snakes, 
and  are  sensitive  to  local  minima  and  noise,  (b)  The  initial  contour  should  be  placed  at  completely 
exterior  or  interior  of  the  boundary  of  the  object  because  the  speed  function  is  explicitly  controlled  by  a 
constant  v  evolving  the  initial  contour  towards  one  direction. 

In  Figure  1,  we  show  how  geometric  active  contours  work  in  a  synthetic  gray  image.  Due  to  the  level  set 
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implementation,  the  model  allows  automatic  change  of  topology.  In  Figure  2,  we  show  how  sensitive 
geometric  active  contours  are  to  the  initial  condition  and  noise.  An  initial  contour  is  placed  at  the 
exterior  of  the  ellipse  partially  overlapping  the  triangle.  The  contour  cannot  detect  the  triangle  because 
the  initial  contour  is  not  located  at  the  complete  exterior  of  the  triangle.  Due  to  noise,  the  contour  cannot 
stop  on  the  boundary  of  the  object. 


Initial  Contours 


Deformed  contour 

A 


Figure  1  Geometric  active  contours  applied  into  a  synthetic  gray  image  (256x256x1).  (Left)  An  initial 
contour  is  placed  at  the  exterior  of  all  objects.  (Middle)  The  contour  splits  and  stops  on  the  boundary  of  the 
objects.  (Right)  The  image  is  divided  into  two  segments;  objects  and  the  background. 


Initial  Contours 


Deformed  contour 


Final  segmentation 


Figure  2  Geometric  active  contours  applied  into  a  synthetic  gray  image  (256x256x1)  corrupted  by  noise. 
(Left)  Initial  contour  is  not  placed  at  the  complete  exterior  of  all  objects.  (Middle,  Right)  The  contour 
cannot  stop  on  the  boundary  of  the  object. 

Let  us  define  a  vector- valued  image  function  as 
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l{x,y):R^ 


(14) 


with  components 

^  M  (15) 

where  m  =  1,  2,  ...,  M.  An  image  function  /  at  a  given  coordinate  (x,y)  is  an  Mxl  vector.  We  can 
implement  geometric  active  contour  models  into  vector-valued  images  by  using  a  modified  version  of 
the  image  gradient  operator.  Taking  the  average  of  the  image  gradient  measured  at  each  frame  is  an 
example  and  given  by 

M 

=  (16) 

m 

where  is  an  element  of  a  normalized  weight  vector  to  control  the  relative  importance  of  frames. 

In  Figure  3,  three  circles  with  different  colors  are  placed  in  a  synthetic  RGB  image  (256x256x3).  In 
Figure  4,  we  show  how  geometric  active  contours  work  on  the  image  in  Figure  3.  The  contour  stops  on 
the  external  boundary  of  the  three  circles  after  initilization  at  the  exterior  of  the  three  circles.  Since  the 
segmentation  relies  on  the  deformation  of  a  single  contour,  we  cannot  find  more  than  two  segments 
simultaneously.  In  Figure  5,  we  show  how  we  can  detect  the  internal  boundary  of  an  object.  A  contour  is 
initiated  at  the  interior  of  one  circle,  and  moves  outward.  The  contour  stops  growing  on  the  internal 
boundary  of  the  circle. 


Figure  3  Three  circles  are  located  in  a  synthetic  RGB  image  (256x256x3).  (Left)  Red  channel,  (Middle) 
Green  channel,  (Right)  Blue  channel. 
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Initial  Contours  Deformed  contour  Final  segmentation 


Figure  4  Geometric  active  contours  applied  into  Figure  3  (256x256x3).  (Left)  The  initial  contour  is  placed  at 
the  exterior  of  three  circles.  (Middle)  The  contour  stops  on  the  external  boundary  of  the  three  circles. 
(Right)  The  image  is  divided  into  two  segments;  three  circles  as  one  object  and  the  background. 


Initial  Contours  Deformed  contour  Final  segmentation 


Figure  5  Geometric  active  contours  applied  into  Figure  3  (256x256x3),  corrupted  by  noise.  W=[l  2  1]^  (Left) 
A  contour  is  initiated  at  the  interior  of  one  circle.  (Middle)  The  contour  stops  on  the  internal  boundary  of 
the  object.  (Right)  The  image  is  divided  into  two  segments;  a  circle  and  the  background. 


In  Figure  6,  a  toy  tank  is  placed  in  a  homogeneous  background.  In  Figure  7,  the  same  tank  is  placed  in  a 
textured  background.  Both  images  were  measured  at  the  inside  of  a  room  with  three  channels  (RGB). 
Figure  6  represents  a  simple  real  image,  while  Figure  7  represents  a  more  complicated  real  image  with  a 
textured  background.  In  Figure  8,  we  show  how  geometric  active  contours  work  on  the  image  in  Figure 
6.  After  initiating  an  initial  contour  at  the  exterior  of  the  object,  the  contour  stops  at  the  boundary  of  the 
object  providing  a  silhouette  of  the  object.  In  Figure  9,  we  show  how  geometric  active  contours  work  on 
the  image  in  Figure  7.  Due  to  the  textured  background,  the  stopping  function  detects  high  gradient  not 
only  on  the  boundary  of  the  object  but  also  on  the  background.  The  texture  of  camouflage  paint  also 
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produces  false  edge  pixels  inside  of  the  object.  The  contour  keeps  shrinking  even  after  crossing  the 
boundary  of  the  object. 


Figure  6  A  toy  tank  is  placed  in  a  homogeneous  background  in  an  RGB  image  (258x253x3).  (Left)  Red 
channel,  (Middle)  Green  channel,  (Right)  Blue  channel. 


Figure  7  A  toy  tank  is  placed  in  a  textured  background  in  an  RGB  image  (258x253x3).  (Left)  Red  channel, 
(Middle)  Green  channel,  (Right)  Blue  channel. 


Figure  8  Geometric  active  contours  applied  into  Figure  6.  (Left)  An  initial  contour  is  placed  at  the  exterior 
of  the  object.  (Middle)  The  contour  stops  at  the  boundary  of  the  object.  (Right)  The  image  is  divided  into 
two  segments;  the  object  and  the  background. 
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Initial  Contours  Deformed  contour  Final  segmentation 


Figure  9  Geometric  active  contours  applied  into  Figure  7.  (Left)  The  initial  contour  is  placed  at  the  exterior 
of  the  object.  (Middle)  (Right)  The  contour  keeps  shrinking  even  after  crossing  the  boundary  of  the  object. 
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Geodesic  active  contour 

The  geodesic  active  contour  model  was  proposed  by  Caselles  et  al.  in  [3,  4].  They  considered  a 
particular  case  of  the  snake  energy  function  in  equation  (2),  where  /^=  0  and  image  gradient  replaced 
with  a  decreasing  gradient  function 


E[C)  =  aj|C'(^)p  ds  +  X^g{^l)ds 

0  0 

^E„(CyE,JC) 


(17) 


Based  on  classical  dynamical  systems  principles,  minimizing  this  energy  function  in  a  certain  allowed 
space  of  curves  is  equivalent  to  minimizing 

L{C) 

Lr  =  \  g[l(c{s)))dv,  (18) 

0 


where  dv  is  the  Euclidean  arc  length  [23].  Therefore,  solving  the  active  contour  problem  is  equivalent  to 
finding  a  path  of  minimal  distance,  where  the  distance  is  given  by  the  modified  arc  length  gdv.  This  path 
is  called  geodesic  curve,  and  obtained  by  minimizing  Zr.  The  flow  minimizing  Lr  is  given  by 

dC 

—  =  g(/)/ai-(Vg*n)n,  (19) 

ot 

where  k  is  the  mean  curvature  in  equation  (8)  and  n  is  the  unit  inward  normal  vector  in  equation  (7). 
The  complete  geometric  interpretation  of  this  flow  is  discussed  in  [4,  22].  From  the  relation  between  a 
contour  and  a  level  set  function  defined  in  equation  (4)  and  the  level  set  formulation  of  the  steepest 
descent  method,  solving  this  geodesic  problem  is  equivalent  to  searching  for  the  steady  state  of  the 
evolution  equation 


dcj) 

dt 


div 


v<t>\ 

g{i)K\vci\+vg{iyv(t> 


(20) 


In  order  to  increase  the  convergence  speed,  we  can  add  a  positive  constant  term  v  into  the  evolution 
equation 


d(/) 

77 


div 


^7)^]|V<^|  +  vg(/)|V<^i| 


g{i){K+v)\V(t>\+vg{iyv(t> 


(21) 
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This  is  the  formulation  of  geodesic  active  contour.  The  role  of  v  is  to  increase  the  convergence  speed  by 
minimizing  the  enclosed  area  [9].  We  can  notice  the  geodesic  active  contour  model  is  identical  to  the 
geometric  active  contour  model  in  equation  (13)  except  for  Vg-V^. 

In  Figure  10,  we  show  how  geodesic  active  contours  work  on  the  same  image  that  we  used  for  the 
geometric  active  contours  in  Figure  1.  As  the  initiated  contour  moves  towards  the  boundary  of  objects,  a 
set  of  contours  automatically  appear  from  the  opposite  side  (inside  for  this  case)  and  then  move  towards 
the  boundary  of  objects.  These  dual  active  contours  help  the  initiated  contour  not  to  across  the  weak 
edge  pixels.  The  image  is  divided  into  two  segments,  the  boundary  of  objects  and  the  background. 


Initial  Contours 


Deformed  contour  Final  segmentation 


Figure  10  Geodesic  active  contours  applied  into  a  synthetic  gray  image  (256x256x1).  (Left)  A  contour  is 
initiated  at  the  exterior  of  all  objects.  (Middle)  As  the  contour  shrinks,  a  set  of  contours  appear  from  the 
inside  of  objects  and  move  towards  the  boundary  of  objects.  (Right)  The  image  is  divided  into  two 
segments;  the  boundary  of  objects  and  the  background. 


Color  snakes 

Color  snakes,  a  particular  case  of  geodesic  active  contours  on  vector-valued  images  were  proposed  by 
Sapiro  [22,  23].  He  defined  the  edges  in  vector-valued  images  based  on  classical  Riemannian  geometry. 
The  difference  of  an  image  function  at  two  points  P  and  Q  is  given  by 

M  =  I{P)-I{Q).  (22) 

When  the  Euclidian  distance  between  the  two  points  d{P,  Q)  tends  to  zero,  the  difference  A/  becomes 
the  arc  length  element 

dl  dl 

dl  =  — dx-\ - dy  .  (23) 

dx  dy 
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Its  squared  norm  is  called  the  first  fundamental  form  [12]  and  given  by 


dr  = 

dx 

T 

§11 

§12 

dx 

dy 

.§21 

§22  _ 

dy 

using  the  notation  of  Riemanian  geometry  where 


(24) 


dl  dl  ^  dl  dl 

gl2  =§21  =  - ^ 

OX  oy  ox  cy 


§11 


dl  dl 
dx  dx 


M 


=z 


dl. 


m=\  \ 
M  f 


dl  dl  ^ 

^22  ~  o  ~  Z—t 

Sy  dy 


dx 

V 


(25) 


The  extrema  of  this  quadratic  form  are  obtained  in  the  directions  of  the  eigenvectors  of  the  metric  tensor 
[gij],  and  the  values  of  the  extrema  are  the  corresponding  eigenvalues  given  by 


§11  +  §22  ±  V(§ll-§22f  +4§12" 


(26) 


The  eigenvector  provides  the  direction  of  maximal  and  minimal  changes  at  a  given  point  in  the  image, 
and  the  two  eigenvalues  are  the  corresponding  rates  of  changes,  respectively  X+  and  X..  In  single  frame 
images,  X+  =  |VI|^  and  X.  =  0.  The  strength  of  edges  in  vector-valued  images  is  a  function  of  two 
eigenvalues  X+  and  X.  rather  than  just  the  maximal  rate  of  changes.  A  common  choice  of  the  edge 
function  is 

=  (27) 

Geodesic  active  contour  model  (21)  with  this  particular  color  gradient  operator  constitutes  the  color 
snakes  model. 


In  Figure  11,  we  show  how  color  snakes  work  on  the  same  synthetic  RGB  image  (256x256x3), 
corrupted  by  noise  this  time,  we  used  to  evaluate  geometric  active  contours  in  Figure  5.  Compared  to 
geometric  active  contours,  we  can  notice  that  color  snakes  can  detect  weaker  edge  pixels,  but  are 
sensitive  to  noise. 
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Initial  Contours  Deformed  contour  Final  segmentation 


Figure  11  Color  snakes  applied  into  the  RGB  image  in  Figure  3  (256x256x3),  corrupted  by  noise.  (Left)  A 
contour  is  initiated  at  the  exterior  of  the  three  circles.  (Middle)  (Right)  Color  snakes  find  the  boundary  of 
three  circles  as  well  as  many  false  segments. 


In  Figure  12  and  Figure  13,  we  show  how  color  snakes  work  on  the  same  real  RGB  images  we  used  to 
evaluate  geometric  active  contours  in  Figure  8  and  Figure  9.  Color  snakes  find  the  many  internal 
boundaries  because  of  camouflage  paint  in  both  Figure  12  and  Figure  13.  Color  snakes  produces  many 
false  segments  because  of  the  textured  background  in  Figure  13. 


Initial  Contours 


Final  segmentation 


Figure  12  Color  snakes  applied  into  the  RGB  image  in  Figure  6  (258x253x3).  (Left)  A  contour  is  initiated  at 
the  exterior  of  the  object.  (Middle)  (Right)  Color  snakes  find  many  internal  boundaries  because  of 
camouflage  paint. 
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Initial  Contours  Deformed  contour  Final  segmentation 


Figure  13  Color  snakes  applied  into  the  RGB  image  in  Figure  7  (258x253x3).  (Left)  A  contour  is  initiated  at 
the  exterior  of  the  object.  (Middle)  (Right)  Color  snakes  produces  many  false  segments  and  internal 
boundaries  because  of  the  textured  background  and  camouflage  paint. 
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Discussion 

We  have  implemented  two  boundary-based  active  contour  models;  geometric  active  contours  and 
geodesic  active  contours.  Color  snakes  are  a  particular  case  of  geodesic  active  contours  for  vector¬ 
valued  images. 

After  numerical  experiments,  we  found  a  few  problems  of  the  boundary-based  active  contour  models, 
(a)  Due  to  the  structure  of  the  speed  function,  boundary -based  active  contour  models  evolve  the  contour 
towards  one  direction.  Therefore,  an  initial  contour  should  be  placed  at  completely  exterior  or  interior  of 
the  boundary  of  the  object,  (b)  Due  to  the  image  gradient  operator,  boundary-based  active  contour 
models  may  produce  false  segments  in  noisy  or  textured  images,  (c)  Boundary-based  active  contour 
models  cannot  detect  more  than  two  segments  simultaneously  as  they  rely  on  single  deformable  contour. 
In  Figure  4,  all  circles  are  classified  into  one  object  though  each  circle  has  different  properties  (color). 

The  sensitivity  to  initial  condition  and  noise  can  be  improved  by  a  different  approach  of  active  contours 
-  region-based  active  contour  -  which  will  be  introduced  in  the  next  section.  The  problem  of  textured 
image  can  be  solved  by  the  texture  based  active  contours,  another  form  of  active  contours  on  vector¬ 
valued  images,  particularly  a  stack  of  filter  responses.  The  problem  of  multiple  segments  can  be  solved 
by  multi-phase  active  contours.  The  multi-phase  active  contours  will  be  discussed  after  the  region-based 
active  contours  in  this  report. 
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Statistics  (region)  based  active  contours 

Statistics-  or  region-based  active  contour  methods  move  contours  using  forces  estimated  from  stochastic 
information,  i.e.  mean,  variance,  of  each  region  instead  of  geometrical  boundaries.  An  objective 
function  is  minimized  by  looking  for  the  stochastic  information,  which  characterizes  the  corresponding 
region,  and  the  objective  function  acts  as  the  stopping  function  without  the  image  gradient.  Two  region- 
based  active  contour  models,  one  proposed  by  Chan  and  Vese  [5,  6,  7,  8,  17,  25],  and  the  other  proposed 
by  Rousson  and  Deriche  [17,  20,  21],  will  be  discussed  in  this  section. 


Mumford-Shah  segmentation  model 


Let  us  consider  both  an  image  as  a  set. 


Q  c  (28) 

which  consists  of  finite  number  of  subsets,  and  let  a  contour  C  be  a  closed  subset  (a  contour)  in  Q  made 

up  of  a  finite  set  of  curves.  The  connected  components  of  Q\C  are  defined  as  Qi,  such  that 

f  \ 


Q  = 


UC. 


(29) 


V  /  J 

We  define  |C|  as  the  total  length  of  a  curve  C  and  Io(v,y)  as  a  given  image  function.  In  [15],  Mumford 
and  Shah  defined  the  segmentation  problem  as  follows:  find  a  decomposition  Qi  of  Q  and  an  optimal 
piecewise  smooth  approximation  I(x,y)  of  the  given  image  Io(x,y),  such  that  I(x,y)  varies  smoothly 
within  each  Qi,  and  discontinuously  across  the  boundaries  of  Qi.  They  also  proposed  the  following 
energy  function"^ 


F“(/,C)  =  dxdy  +  dxdy  +  v\C\  (30) 


where  //,  v  are  positive  constants  to  weight  different  terms  in  the  energy.  l(xy)  is  the  minimizer  of  the 
above  energy  and  an  optimal  piecewise  smooth  approximation  of  the  initial,  possibly  noisy,  image  Iq 
(xy).  C  has  the  role  of  approximating  the  edges  of  Io(x,y);  I(x,y)  will  be  smooth  only  the  outside  of  C, 
i.e.  Q\C. 


Level  set  method  for  Mumford-Shah  model 

Let  us  define  the  given  curve  C  as  the  boundary  of  an  open  set  co 

C  =  d(o,  coeQ,  (31) 


^  We  change  the  notation  of  the  given  image  function  I(x,y)  in  order  to  follow  the  notation  in  references. 
Note  I(x,y)  is  approximated  at  each  iteration  (time  varying),  while  Io(x,y)  is  measured  at  only  the  first 
time  (fixed).  The  Io(x,y)  denotes  the  I(x,y)  introduced  in  previous  sections. 
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and  as  a  zero  level  set  of  a  continuous  function  (jix.y)  [25]. 


<j)[x,y)>0  [x,y)&o} 

<^[x,y)  =  0  (x,y)ed(U  (32) 

^{x,y^<0  (x,j)eQ\&> 


A  common  way  to  define  a  level  set  function  is  the  signed  distance  to  the  curve  C.  In  [10],  Evans 
and  Gariepy  expressed  the  length  of  the  curve  C  and  the  area  of  a  set  co  using  Heaviside  function 


(O  = 


(34) 


where  Heaviside  function  is  a  unit  step  function  given  by 

,  .  f  1,  z  >  0 

H{z)  =  \  .  (35) 

^  ^  [O,  z<0 

Since  the  unit  step  function  is  not  differentiable,  a  regularized  form  of  Heaviside  function  is  given  by 

(36) 


(37) 


1 

2  I 

— 

iH — arctan 

- 

2 

I  ^ 

\s)) 

The  boundary  of  Heaviside  function  is  a  Dirac  delta  function  approximated  by 

dz  n  +  z^ 


The  associated  Euler-Lagrange  equation  obtained  by  minimizing  the  length  of  curve  in  equation  (33) 
with  respect  to  (f)  is  given  by  [25] 


—  =  (5  (^zi)div 
dt  "  ^  ’ 


V(l) 


(38) 


K 


The  curve  evolution  motivated  by  the  above  equation  can  be  interpreted  as  the  motion  by  mean 
curvature  minimizing  the  length  of  the  curve. 
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Active  contours  without  edges;  Chan-Vese  model^ 

A  reduced  case  of  the  Mumford-Shah  energy  function  is  obtained  by  restricting  the  segmented  image 
I(x,y)  to  a  function,  constant  inside  each  connected  component  Qi,  i.e.  I(x,y)  =  Ci.  The  simplified  energy 
function  is  given  by 


C)  =  X  p  -  c, )'  dxdy  +  V |C| .  (39) 

i  ' 

Assuming  C  is  fixed,  the  above  energy  is  minimized  by  setting 

c.  =  meani^I^  (x,y)),  (x,j;)eQ,..  (40) 

Chan  and  Vese  proposed  an  active  contour  model  using  this  Mumford-Shah  piecewise  constant  model 
[7],  which  does  not  need  an  edge  function.  They  proposed  to  minimize  the  following  energy  function 
with  respect  to  C\,  and  C. 

Fj  (q , Cj, C)  =  £(/o  (x, y) - q  f  dxdy  +  y) “ ^2  f  dxdy  +  v|C|  (41) 

The  above  energy  function  can  be  rewritten  using  the  level  set  formulation  and  Heaviside  function  as 
Fi  (Ci ,  Cj , <2^)  =  (/o  j)  -  q  f  H  {(p)  dxdy 


,  y )  -  Cj  {1  dxdy  . 


(42) 


The  minimum  of  this  energy  function  can  be  obtained  by  the  evolution  equation 


—  =  s  U) 

dt  " 


vdiv 


J 


(43) 


where 

L  ^0  y))dxdy 

c  =  - 

\^dl,((l){x,y))dxdy 


(44) 


^  Although  the  actual  title  of  the  paper  written  by  Chan  and  Vese  is  ‘Active  Contours  without  Edges’, 
there  are  other  active  contour  models  which  do  not  use  an  edge  function.  Thus,  we  will  call  this  model 
as  ‘Chan-Vese  model’  following  the  names  of  the  authors. 
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(45) 


^2 


||  dxdy 

J 

u 

((*( 

x,y)) 

||  dxdy 

This  model  performs  active  contours  looking  for  a  2-phase  segmented  image,  given  by 

I  {x,y)  =  c,H  [(l){x,y))  +  c^[\-  H  [(l){x,y))Y 


(46) 


The  main  advantages  of  this  active  contour  model  compared  to  other  active  contour  models  are;  (a)  It 
automatically  detects  interior  contours,  so  we  can  place  a  random  number  of  initial  contours  in  random 
places  in  the  image.  There  are  two  advantages  of  this  distributed  seed  initialization.  First,  we  do  not 
need  prior  knowledge  of  the  possible  location  of  the  object.  Second,  the  convergence  speed  increases  as 
the  number  of  initial  contours  increases,  (b)  Since  this  active  contour  model  does  not  need  an  edge 
function,  objects  without  edge  lines  can  be  detected,  (c)  Multiple  contours  can  be  evolved 
simultaneously  providing  multiple  segments. 


In  Figure  14,  we  show  how  Chan-Vese  model  works  on  the  same  synthetic  gray  image  (256x256x1) 
used  to  evaluate  other  models.  36  small  initial  contours  were  placed  at  random  places  in  the  image. 
Compared  to  other  active  contour  models,  Chan-Vese  model  is  robust  respect  to  noise. 


Initial  Contours  Final  segmentation 


Figure  14  Chan-Vese  model  applied  into  a  synthetic  gray  image  (256x256x1)  corrupted  by  noise.  (Left)  36 
small  initial  contours  are  placed  using  distributed  seeds  in  the  image.  (Middle)  (Right)  The  image  is  divided 
into  two  segments;  objects  and  the  background. 


Figure  15  is  a  long  wavelength  infrared  (LWIR)  image  measured  outside.  This  LWIR  image  has  low 
resolution  and  contrast.  A  toy  truck  was  placed  on  grassland  during  a  sunny  day.  The  cargo  bed  of  the 
truck  is  made  of  steel,  while  other  parts  of  the  truck  are  made  of  plastic.  In  Figure  16,  we  show  the  result 
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of  Chan-Vese  model  applied  into  Figure  15.36  initial  contours  are  placed  using  distributed  seeds.  Active 
contours  produce  a  few  false  segments  because  of  the  textured  background. 


Figure  15  An  outdoor  LWIR  image  (256x256x1)  A  toy  truck,  mainly  consisting  of  plastic  parts  and  metal 
parts,  is  placed  on  grassland. 


Initial  Contours  Final  segmentation 


Figure  16  Chan-Vese  model  applied  into  Figure  15.  (Left)  36  initial  contours  are  placed  using  distributed 
seeds.  (Middle)  (Right)  A  few  false  segments  are  produced  because  of  the  texture  of  grass.  The  image  is 
divided  into  two  segments;  object  and  the  background. 

In  [7],  Chan  et  al.  proposed  the  extension  of  their  active  contour  model  to  vector-valued  images.  Their 
model  takes  an  average  of  energy  functions  measured  at  each  frame.  The  energy  function  and  evolution 
equation  for  vector- valued  images  are  given  by 


6 


The  contrast  of  this  image  is  enhanced  to  visualize  the  shape  of  the  object. 
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(47) 


Fim  (ci ,  C2 ,  ^^)  =  I;  (/o„  {x,y)-  Ci„  )"  H  {(f)  dxdy  + 

m=l 

^  r  2 

Z  (^0«  j)  -  )  {1  -  ^  + 

m=l 

vJJVH(^)| 


5? 


vdiv 


N. 


(48) 


where  m=  1,2,  . . . ,  M  denotes  the  frame  of  a  given  image,  wim  and  W2m  are  positive  constant  elements  of 
normalized  weight  vectors  to  control  the  relative  importance  of  frames.  The  constant  functions  Ci  and  C2 
are  vectors  in  this  form. 


In  Figure  17,  we  show  the  result  of  Chan-Vese  model  applied  into  the  RGB  image  in  Figure  6 
(258x253x3).  36  initial  contours  are  placed  using  distributed  seeds.  In  Figure  17,  we  can  see  more  detail 
of  the  barrel  of  the  tank  than  Figure  8,  obtained  by  the  geometric  active  contour  model.  In  Figure  18  and 
Figure  19,  we  show  the  result  of  Chan-Vese  model  applied  into  the  RGB  image  in  Figure  7  (258x253x3). 
Active  contours  produce  many  false  segments  because  of  the  textured  background.  In  Figure  19,  the 
number  of  false  segments  is  reduced  by  placing  one  initial  contour  manually  instead  of  distributed  seeds. 


Initial  Contours  Final  segmentation 


Figure  17  Chan-Vese  model  applied  into  the  RGB  image  in  Figure  6  (258x253x3).  (Left)  36  initial  contours 
are  placed  using  distributed  seeds.  (Middle)  Active  contours  move  towards  the  boundary  of  the  object  and 
the  boundary  of  the  image.  (Right)  The  image  is  divided  into  two  segments;  the  object  and  the  background. 


27 


Initial  Contours 


Final  segmentation 


Figure  18  Chan-Vese  model  applied  into  the  RGB  image  in  Figure  7  (258x253x3).  (Left)  36  initial  contours 
are  placed  using  distributed  seeds.  (Middle)  (Right)  Active  contours  produce  many  false  segments  because 
of  the  textured  background. 


Initial  Contours  Deformed  contour  Final  segmentation 


Figure  19  Chan-Vese  model  applied  into  the  RGB  image  in  Figure  7  (258x253x3).  (Left)  One  initial  contour 
is  placed  manually.  (Middle)  (Right)  Active  contours  produce  fewer  false  segments  than  Figure  18. 
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Multi-phase  active  contours 

Multiple  objects  with  different  properties  were  given  in  Figure  4  and  Figure  5,  but  we  could  detect  only 
one  object  at  a  time  because  the  segmentation  was  done  by  one  active  contour.  Chan  and  Vese  [5,  6,  17, 
25],  Rousson  and  Deriche  [17,  21],  and  Paragios  [19]  proposed  multi-phase^  active  contour  models  to 
detect  multiple  segments  simultaneously.  In  multi-phase  active  contour  models,  we  deform  multiple 
contours  simultaneously  and  take  their  combination,  usually  the  union,  to  construct  the  final  contour, 
which  divides  the  image  into  more  than  two  segments.  The  final  contour  can  be  given  by 

C  =  UC,  (49) 

j 

where  each  Cj  is  a  closed  curve  (contour). 

Multi-phase  Chan-Vese  model 

In  [5,  6,  17,  25],  Chan  and  Vese  proposed  an  extension  of  the  piecewise  constant  model  into  multi-phase 
segmentation.  They  defined  N  =  2^  segments  using  J  level  set  functions.  With  this  definition,  segments 
form  a  disjoint  decomposition  covering  the  domain  Q.  There  is  no  vacuum  and  overlap  among  segments, 
and  we  need  only  log2V  level  set  functions  to  present  N  segments.  Let  us  consider  a  vector  level  set 
function 

=  (l)2{x,y)  (50) 

where  log2V  is  the  number  of  level  set  functions,  and  each  level  set  function  is  defined  as 

^2)^.  :Q^M.  (51) 

The  union  of  the  zero  level  sets  of  represents  the  edges  in  the  segmented  image.  Let  us  define  a  vector 
Heaviside  function  in  the  same  way 

//(<1))  =  [//W)  -  //(A)]'-  (52) 

Two  pixels  (xi,  yi)  and  (x2,  yi)  belong  to  the  same  segment  if  and  only  if  the  corresponding  Heaviside 
functions  have  the  same  value. 


^  ‘Phase’  is  used  as  an  equivalent  term  to  segment.  Note  ‘multi-phase  active  contours’  denotes  active 
contours  with  more  than  two  phases  instead  of  one  phase.  Although  this  definition  is  confusing,  we 
followed  the  notation  in  references. 
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(53) 


(^1  ’ H  e  Q,  ))  =  // [(!> (xj ,  J2 )) 

y^i^yi )  j 

The  energy  function  of  multi-phase  Mumford-Shah  piecewise-constant  model  can  be  rewritten  from  2- 
phase  model  in  equation  (41)  to 

Fr{c,c)=  E 

/=0 

using  a  constant  vector  of  averages 

c  =  [ci  C2  c^f.  (55) 


,y)-cy  Xidxdy  +  ^\VXi\ 


(54) 


Xi  is  the  segment  function  denoting  whether  a  given  pixel  belongs  to  the  segment  Qi  or  not.  The  segment 
functions  of  2-phase  model  (1  level  set  function)  are  given  by^ 


Zo’ 

-\-h; 

.  H,  _ 

(56) 


and  4-phase  model  (2  level  set  functions)  are  given  by 


Xoo 

Zoi 

Hy-H,) 

Zw 

.Xn_ 

(57) 


After  replacing  the  length  of  contour  with  the  sum  of  the  length  of  zero  level  set 

^ENNELKI,  (58) 

(=0  7=1 

the  energy  function  is  given  by 

'v”  (c.  ® ) = Z  L  v)  -  c, )  xA<iy + 1  v  r  1'^";  I  ■  <59) 

/=0  7=1 

For  a  particular  example,  the  energy  function  of  4-phase  model  is  given  by 


Note  we  simplified  the  notation  of  Heaviside  function.  Hj  =  Hj(x,y)  =  H(^(x,y)) 
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F,{c,^)  =  \J^I,{x,y)-c^)\\-H,){\-H,)dxdy 
+ la  (^0  i^’y)~  ^01 )"  (1  -  ^2  )H^dxdy 

+ la  (^0  i^’y)~  ^10 )"  ^2  (1  “  )  d^dy 

+ja(^o  i^’y)~^nf  H^H.dxdy 

+v(JJVi/(^)|  +  JJVi/(4)|) 

where  c  =  {cqo,  Cqi,  Cio,  Cn}  is  a  constant  vector  in  equation  (55)  and  O  =  {^i,  is  a  level  set  vector  in 
equation  (50).  The  segmented  image  function  I(x,3;)  for  4-phase  model  is  given  by 


Two  evolution  equations  are  obtained  by  minimizing  the  above  energy  function  as 

^  (A\  f  1  {Oo-C„)  -(4-Coi)  }^2 

[+{(4-qo)  -(4-Coo)  }(1-^2)J 
{Oo-C„)  -po-Coi)  }^1 

^  =  4(^^2)hdlv  7^  “  r  2  21  r 


where  the  constant  vector  is  estimated  by 


(4(x,j;)),  if(x,j;) 


^^{x,y)<0,<j)^{x,y)<0 

(l)^{x,y)<0,(t)^{x,y)>Q 

^^[x,y)>Q,^^[x,y)<Q 

(l)^{x,y)>Q,(l)^{x,y)>Q 


In  [25],  Vese  proposed  the  extension  of  this  model  to  vector-valued  images  as 

N-\  M  J 


(62) 


(63) 


(64) 


In  Figure  20  and  Figure  21,  we  show  the  results  of  4-phase  Chan- Vese  model  applied  into  a  synthetic 
RGB  image  (256x256x3)  corrupted  by  noise  in  Figure  3.  Four  segments  are  defined  by  two  level  set 
functions.  In  Figure  20,  two  initial  contours  were  placed  manually,  and  the  image  is  divided  into  four 
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segments  classifying  the  three  circles  and  the  background  separately.  In  Figure  21,  36x2=72  initial 
contours  are  placed  using  distributed  seeds,  and  the  image  is  divided  into  3  segments  misclassifying  two 
circles  as  the  same  segment.  From  these  experiments,  we  show  that  Chan-Vese  model  gives  more 
options  to  place  initial  contours,  but  the  segmentation  result  is  still  affected  by  the  initial  conditions. 


Initial  Contours 


Deformed  contour 


Final  segmentation 


Figure  20  4-phase  (2  level  set  functions)  Chan-Vese  model  applied  into  the  synthetic  RGB  image  in  Figure  3 
(256x256x3),  corrupted  by  noise.  (Left)  Two  initial  contours  are  placed  manually.  (Middle)  (Right)  The 
image  is  divided  into  4  segments  {circle  1,  circle  2,  circle  3,  and  the  background}. 
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Initial  Contours 


Deformed  contour 


Final  segmentation 


Figure  21  4-phase  (2  level  set  functions)  Chan-Vese  model  applied  into  the  synthetic  RGB  image  in  Figure  3 
(256x256x3),  corrupted  by  noise.  (Left)  36x2=72  initial  contours  are  placed  using  distributed  seeds.  (Middle) 
(Right)  The  image  is  divided  into  3  segments  misclassifying  two  circles  as  the  same  segment. 


In  Figure  22,  we  show  a  real  multispectral  image.  A  toy  truck  was  placed  on  the  grassland  during  a 
sunny  day.  Seven  images  were  measured  at  different  wavelengths  using  three  different  sensors.  The  first 
three  images  were  measured  by  an  RGB  camera,  and  the  next  three  images  were  measured  by  an  MWIR 
camera  between  3  ~  5  pm,  and  the  last  image  was  measured  by  an  LWIR  camera  between  8  ~  12  pm. 
The  last  image  shows  the  original  version  of  Figure  15.  The  segmentation  of  this  multispectral  image  is 
a  difficult  task  because  (a)  the  background  contains  complicated  texture,  (b)  The  object  consists  of  small 
sub-parts,  and  each  subpart  show  different  properties.  The  cargo  part  is  made  of  yellow  steel,  tire  part  is 
made  of  black  plastic,  and  driving  seat  is  made  of  plastic  with  multiple  colors,  (c)  The  shadow  of  the 
object  makes  the  spectral  distribution  of  pixels  more  varied.  In  Figure  23,  we  show  the  result  of  4-phase 
Chan-Vese  model  applied  into  this  image  set.  Two  sets  of  initial  contours  are  placed  using  distributed 
seeds.  The  segmentation  result  is  not  as  simple  as  other  cases. 
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Figure  22  A  7-band  multispectral  image  {red,  green,  blue,  3~5pm,  3.3~5jam,  3~4.3pm,  8~12pm}  (256x256x7). 
A  toy  truck,  mainly  consisting  of  plastic  parts  and  a  few  metal  parts,  is  placed  on  a  grass  field. 


Figure  23  4-phase  Chan-Vese  model  applied  into  the  multispectral  image  in  Figure  22  (7bands;  3  visible,  3 
MWIR,  1  LWIR). 
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Multi-dimensional  Gaussian  fitting 

In  [17,  21],  Rousson  and  Deriche  proposed  an  active  contour  model  using  the  maximization  of  the  a 
posteriori  segmentation  probability.  Let  pi{I)  be  the  conditional  probability  density  function  (PDF)  of  an 
image  function  value  /  with  respect  to  the  hypothesis  hi.  Rousson  and  Deriche  defined  a  segmentation 
problem  as  solving  the  optimization  problem  with  respect  to  the  a  posteriori  segmentation  probability, 
given  the  observation  set  p{oiO)\l),  where  is  a  segment  of  a  domain  Q.  With  two  assumptions,  (a)  all 
segments  are  equally  possible,  (b)  the  pixels  within  each  region  are  independent;  the  optimal  frame 
segment  is  obtained  by  minimizing  the  following  energy. 

-log p^  {^l{^x,y)^dxdy  +  \C\  (65) 

/=i 

For  a  smooth  non-textured  image,  a  common  choice  is  to  use  Gaussian  distributions.  It  means  that  the 
conditional  probability  with  respect  to  hi  for  a  vector  valued  image  pixel  I  is 


Pi(K^^y)) 


2;r  E 


|l/2 


(66) 


where  and  Si  are  the  mean  vector  and  covariance  matrix  associated  to  the  segment  Qi  of  the  vector 
valued  image  I(x,y).  Disregarding  constant  terms,  the  above  energy  function  can  be  rewritten  as  a 
general  form 

F (C)  =  e^dxdy  +  e^dxdy  +  |C|  (67) 

where  the  objective  function  for  each  segment  is  given  by 

e.  {x,y)  =  log  Is .  I  +  (l  (x,  j)  -  f  S (l  (x,  j)  -  ) .  (68) 


The  solution  of  the  Euler-Lagrange  equations  obtained  from  the  energy  function  gives  expressions  of 
and  Si 


\^i  = 


l{x,y)Zidxdy 

l^Zidxdy 


(69) 


^  (l j)  - 11, ) (l  j)  - 11,  f  Xidxdy 

l^Xidxdy 

where  the  segment  function  Xi  is  identical  to  Chan-Vese  model.  The  evolution  equation  is 


(70) 
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d(j) 

dt 


vdiv 


InJ 


This  active  contour  model  can  be  extended  to  a  multi-phase  model  using  the  same  set  of  segment 
functions  used  for  Chan-Vese  model. 


In  Figure  24  and  Figure  25,  we  show  the  results  of  Rousson-Deriche  model  applied  into  the  RGB  image 
in  Figure  7  (258x253x3).  We  placed  an  initial  contour  manually  in  Figure  24  and  used  distributed  seeds 
in  Figure  25.  In  both  cases,  Rousson-Deriche  model  find  the  boundary  of  the  object  in  a  textured 
background.  Rousson-Deriche  model  is  the  only  active  contour  model,  which  finds  the  boundary  of  the 
object  in  this  image.  The  barrel  of  the  tank  is  not  detected  because  it  is  made  of  different  material;  the 
barrel  is  made  of  black  plastic,  but  other  parts  of  the  tank  is  made  of  metal  with  camouflage  paint.  The 
image  is  divided  into  two  segments;  the  object  and  the  background. 


Initial  Contours  Deformed  contour 


Final  segmentation 


Figure  24  Rousson-Deriche  model  applied  into  Figure  7  (258x253x3).  (Left)  An  initial  contour  is  placed 
manually  at  the  exterior  of  the  object.  (Middle)  The  contour  moves  towards  the  object  and  stops  on  the 
boundary  of  the  object.  (Right)  The  image  is  divided  into  two  sets;  the  object  and  the  background. 
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Initial  Contours  Deformed  contour  Final  segmentation 


Figure  25  Rousson-Deriche  model  applied  into  Figure  7  (258x253x3).  (Left)  9  initial  contours  are  placed 
using  distributed  seeds.  (Middle)  The  contours  move  towards  the  object  and  stops  on  the  boundary  of  the 
object.  (Right)  The  image  is  divided  into  two  sets;  the  object  and  the  background. 
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Discussion 

We  have  implemented  two  region-based  active  contour  models:  Chan-Vese  active  contour  model  based 
on  Mumford-Shah  segmentation  funtion  and  Rousson-Deriche  active  contour  model  based  on  the 
maximization  of  a  posteriori  segmentation  probability.  After  numerical  experiments,  these  two  region- 
based  active  contour  models  show  a  few  advantages  compared  to  the  boundary-based  active  contour 
models;  (a)  region-based  active  contours  use  the  global  energy  minimization  for  stopping  function 
instead  of  an  edge  function  based  on  the  local  image  gradient.  According  to  visual  inspection,  the  global 
minimization  often  shows  more  reasonable  segmentation  result  than  local  minimization.  Not  using  a 
local  edge  function  also  gives  more  robustness  to  noise,  (b)  A  random  number  of  initial  contours  can  be 
placed  at  random  places  in  the  image.  Increasing  the  number  of  initial  contours  increases  the 
convergence  speed,  and  distributed  seeds  provide  a  means  for  an  automatic  segmentation  tool,  (c)  The 
multi-phase  segmentation  makes  it  possible  to  divide  an  image  into  more  than  two  segments. 

Although  the  energy  functions  of  two  models  to  be  minimized  are  similar,  they  calculate  different 
metrics.  Chan-Vese  model  measures  a  squared  Euclidean  distance  between  the  given  image  function  / 
and  the  estimated  mean  Ci  of  the  corresponding  region.  Rousson-Deriche  model  measures  the 
Mahalanobis  distance,  if  we  ignore  the  log  term.  Another  difference  between  two  models  is  Chan-Vese 
model  takes  the  average  of  energy  functions  measured  at  each  frame,  while  Rousson-Deriche  model  fits 
a  multi-dimensional  Gaussian  distribution.  Chan-Vese  model  can  be  interpreted  as  a  particular  case  of 
Rousson-Deriche  model,  where  the  covariance  matrix  is  fixed  as  an  identity  matrix.  According  to  visual 
inspection,  this  multi-dimensional  approach  often  shows  more  reasonable  segmentation  result. 
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Prime  contours 

Hyperspectral  images  often  contain  more  than  hundred  frames.  Features  which  discriminate  particular 
object  do  not  necessarily  exist  in  all  frames.  For  instance,  a  hot  spot  of  a  car  engine  is  a  discriminating 
feature  in  infrared  bands,  but  almost  invisible  in  visible  bands.  The  color  of  flowers  is  a  discriminating 
feature  in  visible  bands,  but  is  not  a  discriminating  feature  in  infrared  bands. 

Most  active  contour  models  applicable  to  vector-valued  images  are  extended  to  vector-image 
segmentation  after  being  originally  designed  for  two-dimensional  image  segmentation,  i.e.  Chan-Vese 
active  contour  model.  These  active  contours  often  calculate  the  energy  function  by  taking  the  sum  of 
energy  functions  from  each  frame.  The  energy  function  can  be  high  in  some  frames  but  can  be  low  or 
even  zero  in  other  frames.  Summing  up  these  energies  along  the  dimension  of  frames  will  smooth  the 
discriminating  feature,  i.e.  weakening  the  boundary  of  objects. 

In  Figure  26,  we  show  a  synthetic  RGB  image  (256x256x3)  for  this  kind  of  case.  The  background  has 
intensity  {180,  180,  180}.  Hexagon  and  circle  are  visible  in  all  three  bands  with  intensity  {130,  60,  90}. 
There  are  two  pentagons  with  intensity  {130,  60,  30}  placed  inside  of  the  hexagon  and  circle.  These 
pentagons  are  invisible  in  the  red  and  green  channels  because  they  have  the  same  intensity  as  the 
hexagon  and  circle.  Information  useful  to  extract  these  pentagons  is  available  only  in  the  blue  channel. 
Our  objective  is  to  segment  the  two  pentagons  from  the  hexagon  and  circle.  In  Figure  27,  we  show  the 
result  of  the  4-phase  Chan-Vese  model  applied  into  this  vector-valued  image.  Two  pentagons  are  not 
segmented  because  the  energy  function  from  the  blue  channel  is  smoothed  by  averaging. 


Figure  26  Four  objects  are  placed  in  a  synthetic  RGB  image  (256x256x3).  Only  two  objects,  hexagon  and 
circle,  are  visible  in  red  and  green  channels.  Two  pentagons  are  visible  inside  of  the  hexagon  and  circle  only 
in  the  blue  channel. 
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Initial  Contours  Deformed  contour  Final  segmentation 


Figure  27  4-phase  (2  level  set  functions)  Chan-Vese  model  applied  into  Figure  26.  (Left)  Two  sets  of  contours 
are  initiated  using  distributed  seeds.  (Middle)  Both  contours  cannot  detect  the  pentagons.  (Right)  The  image 
is  divided  into  two  segments  only;  {hexagon,  circle}  and  background. 


A  pseudo  multispectral  image  is  generated  from  an  RGB  image  for  this  experiment.  The  first  three 
frames  are  extracted  from  an  RGB  image.  The  fourth  frame  is  generated  by  Gaussian  blurring  with 
radius  1.0  and  adding  1%  uniform  noise  to  the  gray  image  (average  of  the  first  three  channels).  The  fifth 
frame  is  generated  by  Gaussian  blurring  with  radius  2.0  and  adding  3%  uniform  noise  to  the  blue 
channel.  The  sixth  frame  is  generated  by  Gaussian  blurring  2.0  and  adding  3%  uniform  noise  to  the  red 
channel.  The  seventh  frame  is  generated  by  Gaussian  blurring  with  radius  4.0  and  adding  4%  uniform 
noise  to  the  gray  image.  The  image  pixels  around  the  engine  hood  in  the  sixth  and  seventh  frame  are 
20%  lightened  to  make  pseudo  hot  spot.  Our  objective  is  to  segment  the  hot  spot  from  the  car.  Figure  28 
shows  this  pseudo  multispectral  image  with  seven  bands.  In  Figure  29,  we  show  the  result  of  2-phase 
Chan-Vese  model  applied  into  the  image  in  Figure  28.  The  hot  spot  is  not  segmented  because  the 
averaging  operation  smooth  energy  functions  from  sixth  and  seventh  frame. 
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Figure  28  A  pseudo  multispectral  image  (258x253x7).  (Top)  The  original  RGB  image.  (Bottom)  Images 
modified  from  the  original  RGB  image. 


Initial  Contours 


Deformed  contour 


Final  segmentation 


Figure  29  2-phase  Chan-Vese  model  applied  into  the  image  in  Figure  28  (7  bands).  Hot  spot  is  not  segmented. 


Considering  the  properties  of  hyperspectral  images,  if  there  is  a  discriminating  feature  in  band  A,  the 
same  discriminating  feature  may  exist  or  be  strong  in  the  neighbor  bands  of  band  A.  For  active  contours, 
the  energy  function  will  be  kept  strong  enough  if  we  take  the  average  only  within  the  neighbor  bands 
instead  of  the  whole  bands.  Suppose  a  hyperspectral  image  is  given.  We  divide  the  given  hyperspectral 
image  into  a  group  of  multispectral  images  with  lower  number  of  bands,  based  on  the  correlation 
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between  frames,  i.e.  {a  visible  band  image,  a  SWIR  band  image,  a  MWIR  band  image}.  A  set  of  active 
contours  -  the  prime  contours  -  are  evolved  from  each  of  the  multispectral  images.  Final  segmentation  is 
achieved  from  these  prime  contours  using  the  same  means  of  multi-phase  active  contours.  Two  extreme 
results  are  possible.  First,  we  will  have  an  over-segmented  image  if  the  prime  contours  are  highly 
uncorrelated.  Second,  the  result  will  be  identical  to  the  result  using  other  active  contour  models  if  the 
prime  contours  are  completely  correlated.  However,  most  of  cases  will  be  between  these  two  extreme 
results. 


In  Figure  30,  we  show  the  result  of  the  proposed  method  applied  into  the  same  data  used  in  Figure  27. 
Here,  we  assume  that  these  three  channels  belong  to  different  groups,  so  apply  4-phase  Chan-Vese 
model  separately  into  these  three  channels.  The  results  from  red  and  green  channel  are  identical  because 
they  are  completely  correlated,  but  the  result  from  blue  channel  show  different  segmentation.  The  final 
segmented  image  is  represented  by  the  union  of  three  prime  contours.  The  image  is  divided  into  three 
segments;  {two  pentagons,  hexagon  and  circle,  and  background}. 
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Final  segmentation 


Figure  30  Proposed  method  applied  into  Figure  26  (Top)  4-phase  Chan-Vese  model  applied  separately  into 
three  channels.  Initial  contours  are  placed  using  distributed  seeds  in  all  three  cases.  (Middle)  Only  hexagon 
and  circle  are  segmented  in  red  and  green  channels,  but  two  pentagons  as  well  as  hexagon  and  circle  are 
segmented  in  the  blue  channel.  (Bottom)  The  final  result.  The  image  is  divided  into  three  segments;  {Two 
pentagons},  {hexagon,  circle},  and  the  background. 

In  Figure  31,  we  show  the  result  of  proposed  method  applied  into  the  image  in  Figure  28.  First,  we 
divide  the  seven  frames  into  three  groups  [{1,2,3}  {4,5}, {6,7}].  Then,  we  apply  2-phase  Chan-Vese 
model  separately  into  these  three  groups.  36  initial  contours  are  placed  using  distributed  seeds  for  all 
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three  groups.  From  the  first  two  groups,  active  contours  find  only  the  silhouette  of  the  car,  but  they  find 
the  hot  spot  in  the  third  group.  We  represent  the  final  segmented  image  by  taking  the  union  of  these 
three  images. 


Initial  Contours 


Initial  Contours 


Initial  Contours 


Deformed  contour 


Deformed  contour 


Deformed  contour 


Figure  31  Proposed  method  applied  into  Figure  28  (7  bands).  (Top)  2-phase  Chan-Vese  model  applied 
separately  into  three  groups.  36  initial  contours  are  placed  using  distributed  seeds  in  all  three  groups. 
(Middle)  Active  contours  find  only  the  silhouette  of  the  car  in  the  first  two  groups,  but  find  the  hot  spot  in 
the  third  group.  (Bottom)  The  final  result.  The  image  is  divided  into  eight  segments  (almost  four  segments 
by  visual  inspection)  including  the  hot  spot  and  the  silhouette  of  the  car. 
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How  to  define  the  number  of  group  and  which  group  each  frame  belongs  to  should  be  studied.  As  we 
increase  the  number  of  groups,  we  also  increase  the  number  of  level  set  functions  to  calculate.  The 
computational  cost  is  as  high  as  multi-phase  active  contours  with  one  more  dimension. 
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Conclusion  and  future  works 

We  have  done  an  investigation  of  active  contours  as  an  image  segmentation  method  particularly  for 
multispectral  images.  A  research  review  on  published  methods  has  been  performed  with  numerical 
experiments.  Experimental  results  using  selective  models  on  various  synthetic  and  real  images  are 
shown  in  this  report.  The  segmented  results  have  been  evaluated  using  visual  inspection. 

The  integration  of  the  level  set  theory  and  the  mean  curvature  motion  into  active  contours  allowed  for 
automatic  change  of  topology,  such  as  merging  and  breaking,  and  the  calculations  are  made  on  a  fixed 
rectangular  grid.  The  automatic  topological  changes  make  it  possible  to  detect  multiple  objects,  which  is 
not  possible  for  classical  active  contours.  However,  the  level  set  function  increased  the  dimension  of 
data  to  update  resulting  in  higher  computational  cost. 

Depending  on  techniques  used  as  stopping  functions,  we  divided  active  contours  into  two  groups;  (a) 
geometry-  or  boundary-based  models  and  (b)  statistics-  or  region-based  models.  The  boundary-based 
active  contour  models  use  the  image  gradient  to  stop  the  contour  evolution  on  the  desired  boundary. 
Two  boundary-based  active  contour  models  have  been  reviewed;  geometric  active  contours  and 
geodesic  active  contours.  Although  geometric  active  contours  were  not  designed  for  vector- valued 
images,  we  could  apply  them  into  three-channel  images  using  the  average  of  the  image  gradient.  The 
color  snakes  model  was  implemented  as  a  particular  example  of  geodesic  active  contours.  The 
boundary-based  active  contours  encounter  a  few  limitations;  (a)  since  the  stopping  function  use  the 
image  gradient,  boundary-based  active  contours  make  use  of  only  local  information  like  the  classical 
snakes,  and  are  sensitive  to  local  minima  and  noise,  (b)  Due  to  the  fact  that  the  boundary-based  active 
contours  evolve  an  initial  contour  towards  one  direction,  constrained  by  the  curvature  effect,  the  initial 
contour  should  be  placed  completely  exterior  or  interior  of  the  boundary  of  the  object. 

The  region-based  active  contour  models  do  not  use  the  image  gradient.  Two  region-based  active  contour 
models  have  been  reviewed  in  this  report;  Chan-Vese  model  and  Rousson-Deriche  model.  The  region- 
based  active  contours  show  a  few  advantages  compared  to  boundary-based  active  contours;  (a)  since 
region-based  active  contours  do  not  need  an  edge  function,  objects  without  edge  lines  can  be  detected. 
Also,  region-based  active  contours  are  more  robust  with  respect  to  noise,  (b)  region-based  stopping 
function  is  looking  for  the  global  (within  the  corresponding  region)  minimum  instead  of  a  local 
minimum,  (c)  The  initial  contour  can  be  placed  at  anywhere  in  the  image.  The  convergence  speed  can  be 
improved  by  increasing  the  number  of  initial  contours,  (d)  Multiple  contours  can  be  evolved 
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simultaneously  providing  multiple  segments.  According  to  visual  inspection,  region-based  active 
contours  often  produce  more  reasonable  segmentation. 

Chan-Vese  model  uses  Mumford-Shah  segmentation  techniques  for  the  stopping  function.  Chan-Vese 
model  represents  the  segmented  image  as  a  combination  of  constant  vectors.  Because  of  its  simple 
structure,  the  contour  evolution  of  Chan-Vese  model  is  usually  more  than  5  times  faster  than  Rousson- 
Deriche  model,  while  Chan-Vese  model  is  less  applicable  to  complex  images,  particularly  textured 
images.  Rousson-Deriche  model  uses  more  general  formulation  obtained  from  the  maximization  of  the  a 
posteriori  segmentation  probability.  Rousson-Deriche  model  represents  the  segmented  image  as  a 
combination  of  multi-dimensional  Gaussian  distributions  with  adaptive  mean  vectors  and  covariance 
matrices.  The  contour  evolution  of  Rousson-Deriche  model  is  slower  than  Chan-Vese  model,  while 
Rousson-Deriche  model  is  relatively  more  applicable  to  complex  images  because  of  the  multi¬ 
dimensional  (complex)  structure. 

We  did  an  experiment  of  extracting  a  stack  of  prime  active  contours,  which  are  evolved  from  a  group  of 
frames  instead  of  the  whole  frames,  in  order  to  avoid  the  smoothing  effect.  This  experiment  showed  that 
more  discriminating  features  can  be  extracted  from  hyperspectral  images. 

After  literature  survey  and  numerical  experiments,  we  have  concluded  that  region-based  active  contours 
are  more  practical  approaches  than  boundary-based  active  contours  as  a  multispectral  image 
segmentation  tool.  Active  contours  using  multi-dimensional  statistics,  i.e.  multi-dimensional  histogram, 
are  planed  to  study.  We  expect  the  multi-dimensional  statistics  to  provide  better  segmentation  ability 
with  slightly  higher  computational  cost.  A  supervised  active  contour  model  using  histogram  matching  is 
also  planned  to  study. 
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